Fix retry bug in tree model query which ignore the time filter#18179
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #18179 +/- ##
============================================
+ Coverage 42.04% 42.60% +0.56%
Complexity 318 318
============================================
Files 5312 5323 +11
Lines 375102 376815 +1713
Branches 48422 48742 +320
============================================
+ Hits 157693 160548 +2855
+ Misses 217409 216267 -1142 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
There was a problem hiding this comment.
Pull request overview
This PR fixes a retry-safety bug in tree-model query analysis where analysis-time in-place mutations of the parser-owned AST could cause subsequent retry attempts to observe different semantics (notably losing global time filters). It introduces copy-on-write / journaling for tree-model analysis mutations, adjusts planner/execution lifecycle to explicitly begin/commit/rollback analysis attempts, and adds regression coverage for both tree and table model re-analysis.
Changes:
- Add a tree-model
TreeAnalysisMutationJournaland use it to make analysis-time mutations retry-safe (copy-on-write for nested mutable state; non-mutating time predicate extraction + predicate simplification). - Define an analysis-attempt lifecycle in
QueryExecutionand connect it to planners via newIPlannerattempt hooks. - Add unit tests covering repeated analysis, dispatch-retry behavior, and non-mutation guarantees across ORDER BY, LIMIT/OFFSET pushdown, templates, and table-model device queries.
Reviewed changes
Copilot reviewed 26 out of 26 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| iotdb-core/datanode/src/test/java/org/apache/iotdb/db/queryengine/plan/statement/QueryStatementTest.java | Adds ORDER BY copy-on-write regression test for raw vs normalized expression handling. |
| iotdb-core/datanode/src/test/java/org/apache/iotdb/db/queryengine/plan/relational/analyzer/AnalyzerTest.java | Adds table-model SHOW/COUNT DEVICES re-analysis tests and internal ShowDevice traversal-state preservation test. |
| iotdb-core/datanode/src/test/java/org/apache/iotdb/db/queryengine/plan/planner/LogicalPlanBuilderTest.java | Verifies DeviceView planning doesn’t mutate parsed sort items and uses implicit merge keys correctly. |
| iotdb-core/datanode/src/test/java/org/apache/iotdb/db/queryengine/plan/execution/QueryExecutionRetryTest.java | Adds tests for analysis-attempt commit/rollback sequencing across retry success/exhaustion. |
| iotdb-core/datanode/src/test/java/org/apache/iotdb/db/queryengine/plan/analyze/TemplatedAnalyzeRetryTest.java | Adds template-analysis retry-safety tests (count_time(*) and LIMIT/OFFSET pushdown). |
| iotdb-core/datanode/src/test/java/org/apache/iotdb/db/queryengine/plan/analyze/PredicateUtilsTest.java | Adds non-mutating time-predicate extraction and copy-on-write predicate simplification tests. |
| iotdb-core/datanode/src/test/java/org/apache/iotdb/db/queryengine/plan/analyze/AnalyzeTest.java | Adds broad regression coverage ensuring repeated analysis preserves semantics across many tree-model features. |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/statement/component/WhereCondition.java | Adds factory to wrap already-normalized predicates without rebuilding AST. |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/statement/component/OrderByComponent.java | Adds copyOf to enable analysis-time copy-on-write updates while sharing immutable expression nodes. |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/statement/component/GroupByComponent.java | Adds analyze-only setter to avoid renormalizing already-normalized expressions during rollback/restore. |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/sql/ast/ShowDevice.java | Implements copyForAnalysis() to isolate SQL-parsed ShowDevice from analyzer mutations. |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/sql/ast/CountDevice.java | Implements copyForAnalysis() to isolate SQL-parsed CountDevice from analyzer mutations. |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/sql/ast/AbstractQueryDeviceWithCache.java | Adds “parsed-input-only” copy constructor for per-analysis working statements (shares raw WHERE intentionally). |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/analyzer/StatementAnalyzer.java | Uses working copies for SQL device queries while preserving internal ShowDevice prepared traversal state. |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/planner/TreeModelPlanner.java | Wires planner attempt lifecycle to TreeAnalysisMutationJournal. |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/planner/LogicalPlanBuilder.java | Avoids mutating parsed QueryStatement sort keys for DeviceView implicit merge keys; stores merge order parameter in Analysis. |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/planner/IPlanner.java | Adds default analysis-attempt lifecycle hooks (begin/rollback/commit). |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/expression/visitor/predicate/PredicateSimplifier.java | Converts predicate simplification to non-mutating, reconstructing only when needed (copy-on-write). |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/execution/QueryExecution.java | Adds explicit attempt begin/commit/rollback behavior aligned with dispatch retry lifecycle. |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/analyze/TreeAnalysisMutationJournal.java | New journal implementing attempt-scoped undo/CoW for parser-owned tree-model AST mutations. |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/analyze/TemplatedAnalyze.java | Routes analysis-time ORDER BY updates through the mutation journal. |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/analyze/TemplatedAggregationAnalyze.java | Avoids in-place mutation of count_time(*) and journals LIMIT/OFFSET pushdown after eligibility checks. |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/analyze/PredicateUtils.java | Reworks global time predicate extraction to be non-mutating and returns residual predicate + value-filter flag. |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/analyze/ConcatPathRewriter.java | Routes SELECT/GROUP BY/ORDER BY analysis-time rewrites through mutation journal / CoW. |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/analyze/AnalyzeVisitor.java | Integrates journal into semantic-check-derived flags, time filter extraction, ORDER BY updates, and template path. |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/analyze/Analyzer.java | Ensures per-analyze attempt preparation and rollback-on-exception when Analyzer is invoked directly. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.



Description
Problem
The first query dispatch may fail for a retryable reason. In the reported case, the DataNode hosting the region leader had reached the upper limit of 1,000 internal connections.
QueryExecutionthen analyzes the same parsed statement again. However, tree-model analysis previously modified parser-owned AST state in place, including extracting the global time predicate fromWHERE. As a result, the retry could analyze a statement different from the original SQL and lose its time filter. Other analysis-time mutations, such as normalized expressions,ORDER BY,LIMIT/OFFSETpushdown, and template-specific rewrites, had the same retry-safety risk.Changes
TreeAnalysisMutationJournalfor tree-model analysis:ORDER BY;QueryExecution:DEVICEandTIMEsort keys forDeviceViewin planning-owned state instead of appending them to the parsedQueryStatement.SHOW DEVICESandCOUNT DEVICESstatements on per-analysis working copies. Internal schema-fetchShowDevicestatements retain their pre-populated traversal state because that state is request input rather than derived analysis data.WHEREexpressions in table-model working copies. This analysis path treats them as read-only, replaces rewritten roots only on the working statement, and avoids a deep-copy cost on normal one-attempt queries.These changes ensure that every retry observes SQL semantics equivalent to the original statement while limiting copying to state that analysis actually needs to modify.
Tests
Added regression unit tests covering:
AND,OR, andNOTpredicates without mutating input expressions;GROUP BY,HAVING,ORDER BY,SELECT INTO,LIMIT/OFFSET,count_time(*), and template analysis;DeviceViewandSortNodeplanning without modifying or accumulating sort keys in the parsed statement;SHOW DEVICESandCOUNT DEVICES;ShowDevicetraversal state.